\chapter{Conclusions and Future Work}
\label{ch:conc}
This chapter concludes the work of the Gabor-Boosting face recognition, and presents the future work. The first section gives significant findings and achievements throughout the thesis. The second section discusses the observations in the experiments. Following by the conclusion in the third section, the final section proposes some directions for future research aimed at solving the remaining problems.

\section{Achievements}
In this thesis, the Gabor-Boosting face recognition algorithm is proposed, developed and tested. From experiments on the Gabor-Boosting face recognition algorithm, some important findings were:
\begin{itemize}
 \item Face recognition can be introduced as a two-class classification problem. A set of images in training set are divided into two classes, \textit{i.e.}, clients or impostors. A client is the given individual with claimed identity, and impostors are all other persons except the client. This effectively avoids the difficulties of building multi-class classifiers in face recognition.
 \item The Gabor wavelet transform is beneficial for feature extraction, and reflects salient changes between pixel values. This makes the algorithm robust against global luminance varying between images.
 \item The number of Gabor wavelet features can be reduced by neglecting some high frequency features due to small local variations between high-frequency feature's responses.
 \item AdaBoost is excellent at selecting significant features from a large set of features. 
 \item The computation in AdaBoost training can be optimised by setting criteria to reject some features in each iteration. By optimising the AdaBoost training, the computational time has been significantly reduced. With \textit{optimisation 1}, the computational cost has been reduced to around $50\%$ compared to the cost without any optimisation. The \textit{optimisation 2} has produced a saving on the computational cost of $3.44\%$ to $5.55\%$ of the computational time without any optimisation.
 \item The Potsu weak learner satisfies the requirement of AdaBoost, which demands the minimal error over the training data. The Potsu weak learner is fast and accurate due to the simple perceptron prototype and large number of examples.
 \item The AdaBoost training based on the Potsu weak learners displays its superior performance over the AdaBoost algorithm on other weak learners, such as FLD weak learners.
 \item The ensemble classifier made from AdaBoost training is not appropriate for classification, which suffers from the overfitting in the testing result. It is because of inferior training data and classification mechanism of ensemble. Overfitting can be by-passed by either cross-validation or SVM evaluation.
 \item SVM is very good for classification with respect to selected features from AdaBoost training. The overfitting effect is suppressed by using SVMs through fewer support vectors and optimised weighting schemes.
 \item The multi-class Gabor-Boosting face recognition algorithm is based on multi-class weak learner - mPotsu. The mPotsu weak learner consists of multiple Potsu binary weak learners, and outputs a label vector instead of only one exclusive label.
 \item The multi-class Gabor-Boosting training selects significant features for the multi-person recognition purpose. The training algorithm is a variant from \mbox{AdaBoost.M1} with the improvement on multiple labels from \mbox{AdaBoost.M2}.
 \item When using face recognition based on multi-class classifiers, it is better to have loose requirements with $k$-NN classifiers, which give a number of most plausible individuals rather than only one exclusive individual. When $10$ nearest individuals are given, the recognition rate has been achieved as $84.5\%$.
 \item The computational time in computer vision, especially in training, can be greatly reduced by Grid computing technology.
\end{itemize}


\section{Observations}
One phenomenon observed from the experiments is that the ensemble classifier built from AdaBoost training suffers from overfitting. Because AdaBoost over-emphasises noisy examples and has un-optimised weighted voting, it is apt to overfit the training data. By giving a small size of data, the ensemble quickly recognises all positive and a few negative examples in the first few iterations, then excludes these negative examples in following iterations. In addition, curse of dimensionality and selection bias in the dataset lead to overfitting. The overfitting could be by-passed by using cross-validation or using alternative classifiers. To suppress overfitting, the training data is trained by SVM classifiers with the selected features. The Gabor-Boosting face recognition algorithm has achieved an extremely strong recognition performance. Therefore, ensemble classifier from AdaBoost training is not a good choice for classification due to the overfitting effect in this case, but AdaBoost is considered as an excellent algorithm for feature selection.

\section{Conclusions}
In the thesis, a highly accurate appearance-based grey scale front-view face recognition algorithm - Gabor-Boosting face recognition, is presented. The strong recognition performance in the face recognition is highlighted by three key leading edge techniques - \textit{Gabor wavelet transform}, \textit{AdaBoost}, \textit{Support Vector Machine (SVM)} in computer vision, machine learning, and pattern recognition. The Gabor wavelets are of similar shape as the receptive fields of simple cells in the primary visual cortex, and demonstrate significant potential in biometrics such as iris recognition and face recognition. AdaBoost is one of the most successful and popular learning algorithms. The classification is designed to construct a ``strong" classifier from ``weak" learning algorithms. AdaBoost has also been successfully applied in face detection to select features representing human faces. The SVM is a powerful learning algorithm for solving a variety of learning and function estimation problems, such as pattern recognition and regression estimation. The SVM provides non-linear classification by mapping the input into high-dimensional feature space where a special type of hyperplane is constructed. The overall recognition performance is benefited by using these three key components. The Gabor wavelet transform extracts features which describe texture variations in human faces. The AdaBoost algorithm selects the most significant features which represent different individuals. The SVM constructs a hyperplane for classification with respect to selected features.

The Gabor-Boosting face recognition algorithm is robust under small number of example images per subject and selection-bias in training data. In the situation of extremely small number of positive examples, such as in the \mbox{XM2VTS} database, the system has achieved nearly $0$ false positive rate and moderate false negative rate. In the general situation of few positive examples, such as in the \mbox{FERET} database, the system performance has been improved with nearly $0$ false negative rate and $0$ false positive rate, \textit{i.e.}, approximately $100\%$ recognition accuracy under conditions of a sufficient number of examples per class.
Potential applications of the algorithm are possibly in highly secure face authentication systems dedicated to a small group of clients. The systems thus have ``no false detections on impostors'' and ``an appropriate acceptance detection rate on clients''. 

Therefore, it is clear that the successful AdaBoost algorithm in face detection can be transplanted into face recognition, however some modifications are needed to apply due to different specifications.
\begin{itemize}
	\item Face recognition can be performed in the face verification scenario. In face verification, the system only needs to distinguish clients and impostors, so that a multi-class classification issue is changed into two-class classification issue.
	\item The feature extraction is improved by Gabor wavelet transform rather than using Haar features. The Gabor wavelet features provide more precise and accurate description on human face than Haar features, so that they are very suitable for the face recognition purpose.
	\item In the training dataset, the number of client examples, \textit{i.e.}, positive examples, should be sufficiently large. Otherwise, overfitting will be happened.
	\item The ensemble classifier built on weak learners suffers from overfitting which reduce the performance, but the alternative SVM classifier gives nearly perfect performance with the selected features. Hence, the AdaBoost training is excellent in feature selection, whilst the classification is better done by SVMs.  
\end{itemize}
 

%In the AdaBoost feature selection, a heuristic example-based weak learner - Potsu weak learner is proposed. Potsu weak learner takes the perceptron as its prototype, and use a heuristic thresholding technique to learn decision rules from examples in training data. The Potsu weak learner is superior to other weak learners, \textit{e.g.}, Fisher Linear Discriminant (FLD) weak learner, and results a better performance in recognition. 

%The Gabor-Boosting face recognition has been extended into multi-class classification from two-class classification. In the Multi-class Gabor-Boosting face recognition, a multi-class AdaBoost feature selection algorithm and corresponding multi-class Potsu weak learner are designed. Since multi-class classification is a very difficult pattern recognition task, a loose control face recognition scenario is used through combining machine vision perception and human vision perception. A relative high classification rate $84.5\%$ has been achieved with only small number of plausible candidates.

%The training of Gabor-Boosting face recognition has been implemented by using Grid computing. Due to independence between weak learners, the computation on the training is divided into hundreds of parallel pieces corresponding to each node over the Grid. The Grid computing has significantly reduced the computational time.

\section{Future work}
Future work could be extended in many aspects, such as face pre-processing, investigation on the training dataset, exploration of more types of features, improvement on feature selection and more advanced classification techniques, experiments on other databases.

\begin{itemize}
\item In face pre-processing, the coordinates of two pupils on faces are located manually or provided by database ground truth. Although this is conducted in the offline training process, it would be more desirable to make it a fully automatic process for obtaining the location of eyes on face images. Hence, automatic eye detection approaches could be implemented in the face recognition system.
\item In \mbox{Chapter} \ref{ch:binary}, the face recognition suffers from overfitting, which is partially due to small number of positive examples in the training dataset, while the performance of face recognition has greatly increased with increasing on the number of positive examples. An investigation can be done for finding how many examples are sufficient for face recognition to gain a satisfactory recognition rate.
\item Rather than Gabor wavelet features, other types of features may be  explored and used in the proposed algorithm. Local Binary Patterns (LBP) \cite{Ojala2002,Ahonen2004,Ahonen2006} is widely used in face recognition recently, \textit{i.e.}, LBP features representing face images. The different types of feature give comparable results which help to choose the best type of feature for face recognition.
\item The feature selection is done by a discrete version of AdaBoost, in which the weak learners only give two outputs - either positive or negative. The feature selection could be improved by using a real version of AdaBoost \cite{Schapire1999ml}, in which the weak learners are confidence-rated from a real-value space. It is believed that using a real value rather than a boolean value would improve the performance of feature selection.
\item From \mbox{Section} \ref{sec:suppressingoverfitinSVM}, it is observed that there is still redundancy among the selected features. Hence, during feature selection, the redundancy between candidate features and selected features is an important factor to examine whether the candidate features would be selected. Mutual Information \cite{Shen2005,Shen2006} or Float Search \cite{Pudil1994,LiStan2004} could be implemented for calculating the redundancy.
\item The SVM classifier could be improved by Geometric Programming \cite{Mavroforakis2006}. A geometric framework for the SVM classification problem allows existing geometric algorithms to be directly and practically applied to solve not only separable, but also non-separable classification problems both accurately and efficiently.
\item The performance of face recognition could be enhanced by classifier fusion \cite{Lu2003}, in which more classifiers (\mbox{LDA}, Naive Bayes, Neural Network, etc) can be built up separately, and combined into a complex classifier. 
\item Although the Gabor-Boosting is an appearance based approach for face recognition, some model-based approach could be adopted, such as \mbox{EBGM} \cite{Wiskott1997} into the algorithm to improve its performance at the feature selection stage.
\item In the experiments, only two databases - \mbox{XM2VTS} and \mbox{FERET} are tested. To demonstrate the performance of the proposed algorithm, other larger and more realistic databases may be used in testing, such as the Yale database \cite{Georghiades2001} and the PIE database \cite{Sim2003}. 
\end{itemize}
With development of more advanced computer vision technology, more powerful computational competence, and more precise image acquisition, Gabor-Boosting face recognition holds promise to make facial recognition systems more robust in practice. Face recognition is an exciting and challenging research topic, and it is believed that there is a  large market waiting for the research outcomes.
%\section{Summary}
%Automatic face recognition has gained much attention by both the commercial and public sectors as an efficient and resilient recognition technique in biometrics over the past few years. 